Methods 3, Week 6:

Towards Bayesian Multi-Level Modelling

What did we talk about last week?

  • Tuesday
    • Assignment 2 Research Outline
    • Assignment 2 Data Set
    • Step-by-step Regression Modelling on the Project Data
      • Linear Regression
  • Thursday
    • Getting brms to work on your machine
    • Implementing formulas
    • Setting priors
    • Fitting models

What will we talk about today?

  • Step-by-step regression modelling on the Project Data
    • The Art of Multilevel Modelling (MLM)
    • The Art of Prior Elicitation

Multilevel Models (MLMs)

Why use MLMs?

  • Correct inferences: No grouping underestimates standard errors of regression coefficients
    • Higher-level predictor coefficients most affected
  • Group effects: Extent of grouping in individual outcomes and identification of ‘outlying’ groups correspond to upper-level residuals in MLM
  • Simulatenous group effects and group-level predictor estimation
  • Inference to a population of groups
    • In MLMs, sample groups treated as a random sample from a population of groups.
      • Inferences in fixed effects models cannot be made beyond sample groups

Randomness

  • MLMs use randomness to account for the structure and variability in data that arise from nested or hierarchical relationships

Variables

  • Random
    • Assumed to be measured with error (ex: mlu)
    • Values generalizable to a much larger population with a certain probability distribution
    • Number of values small compared to population
  • Fixed
    • Measured without measurement error (ex: gender, diagnosis)
    • Desired generalization to population or other studies is to the same values
    • Variable contains all or most of population values

Effects

  • Random
    • Assumes that an IV is random
    • Used if the IV levels thought to be subset of possible population values
    • Larger standard error (less powerful)
  • Fixed
    • Assumes that an IV is fixed
    • Generalization of results apply to similar IV values in population
    • Smaller standard errors
  • Intercept only models in MLR are equivalent to random effects ANOVA
  • Inclusion of constant effects level-1 predictors in MLR models equivalent to a random effects ANCOVA.

Coefficients

  • Random
    • Only applies to MLR analysis in which intercepts, slopes, and variances can assume to be random
    • Variances of the slopes and intercepts (if allowed to vary across groups) are called varying (random) coefficients.
  • Fixed
    • Invariant across groups (between-group variance set to zero) -The average intercept or slope is referred to as a “constant (fixed) effect.”

Random Seeding

  • Reproducibility.
    • Anyone who re-runs your code will get the exact same outputs.
  • Essential in many tasks
    1. Splitting data into training/validation/test sets
    • Ensure that the data is divided the same way every time the script is ran
    1. Model training
    • Non-deterministic algorithms (different outputs given the same input, e.g. random forest, gradient boosting)
    1. Benchmarking, Model Comparison, Parametrization
    • If you are testing and comparing multiple algorithms or comparing models, it’s important that all versions have same input

Hierachical Linear Modelling with BRMS

  • The brm function: default arguments
    • chains: Number of Markov Chains to use (default: 4)
      • Using more chains allows us to be more confident in fitting the posterior distributions, as each chain has a different starting point
      • Tip: #chains = #computer cores
    • iter: Total number of iterations per chain (default: 2000)
    • inits: Specify initial values for chains (default: ‘Random’)
    • warmup (“burn-in”): number of iterations thrown away at the beginning of an MCMC run (default: floor(iter/2))
    • cores: Number of CPU cores used for chain building
    • family: Distribution type for the likelihood function (default = gaussian)

Formulas

  • Left-hand outcome, right-hand predictors
  • Constant effects added first, followed by varying ones between brackets.
    • Variables right of “|” are grouping variables.
  • Intercepts (“1”) are assumed by default in R, otherwise (“0”).
  • All correlations between each pair of random effects are estimated
    • Use || between two random effects that are not correlated
  • Interactions between two predictors can be created by the : sign or by multiplying them instead of simply adding them
  • Even if you only care about overall effects, you still need to specify the hierarchical structure of your data
    • Dependencies in the data will affect overall estimates by increasing overall uncertainty

Simple regression model

##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ 1 + MOT_MLU 
##    Data: data (Number of observations: 352) 
##   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 4000
## 
## Population-Level Effects: 
##           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept    -1.06      0.24    -1.53    -0.60 1.00     3854     2714
## MOT_MLU       0.78      0.06     0.66     0.90 1.00     3918     2639
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.77      0.03     0.72     0.83 1.00     4005     2880
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate Est.Error      Q2.5     Q97.5
## R2 0.3186164 0.0334731 0.2497849 0.3804363

Simple Regression Model Results

##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ 1 + MOT_MLU 
##    Data: data (Number of observations: 352) 
##   Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 4000
## 
## Population-Level Effects: 
##           Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept    -1.06      0.24    -1.53    -0.60 1.00     3854     2714
## MOT_MLU       0.78      0.06     0.66     0.90 1.00     3918     2639
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.77      0.03     0.72     0.83 1.00     4005     2880
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate Est.Error      Q2.5     Q97.5
## R2 0.3186164 0.0334731 0.2497849 0.3804363

Multiple predictors

multiple_predictors_model <- brms::brm(
  brms::bf(CHI_MLU ~ MOT_MLU + Diagnosis + Visit + Child.ID),
  data = data,
  file = 'data/w6/multiple_predictors',
  chains = 2,
  cores = 2,
  seed = 112
)

Multiple Predictors Results

##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ MOT_MLU + Diagnosis + Visit + Child.ID 
##    Data: data (Number of observations: 352) 
##   Draws: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 2000
## 
## Population-Level Effects: 
##             Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept      -0.80      0.25    -1.31    -0.34 1.00     2607     1722
## MOT_MLU         0.54      0.06     0.41     0.67 1.00     2159     1728
## DiagnosisTD     0.40      0.08     0.24     0.57 1.00     2176     1424
## Visit           0.17      0.02     0.12     0.21 1.00     2161     1398
## Child.ID       -0.00      0.00    -0.01     0.00 1.00     2622     1613
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.71      0.03     0.66     0.76 1.00     2293     1354
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate  Est.Error      Q2.5     Q97.5
## R2 0.4319807 0.03094327 0.3701946 0.4901642

Multiple Interacting Predictors

multiple_interacting_predictors_model <- brms::brm(
  brms::bf(CHI_MLU ~ MOT_MLU*Diagnosis + Visit + MOT_MLU:Visit + Diagnosis:Visit),
  data = data,
  file = 'data/w6/multiple_interacting_predictors',
  chains = 2,
  cores = 2,
  seed = 112
)

Multiple Interacting Predictors Results

##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ MOT_MLU * Diagnosis + Visit + MOT_MLU:Visit + Diagnosis:Visit 
##    Data: data (Number of observations: 352) 
##   Draws: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 2000
## 
## Population-Level Effects: 
##                     Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept              -0.50      0.50    -1.48     0.50 1.00     1070     1155
## MOT_MLU                 0.54      0.14     0.25     0.83 1.00     1021     1167
## DiagnosisTD            -0.36      0.51    -1.34     0.66 1.00     1196     1073
## Visit                   0.06      0.13    -0.21     0.32 1.00     1035      975
## MOT_MLU:DiagnosisTD    -0.01      0.13    -0.28     0.25 1.00     1169     1036
## MOT_MLU:Visit          -0.00      0.04    -0.07     0.07 1.00      971      985
## DiagnosisTD:Visit       0.24      0.05     0.14     0.33 1.00     1721     1279
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.68      0.03     0.63     0.74 1.00     1786     1302
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate  Est.Error      Q2.5     Q97.5
## R2 0.4732861 0.02745144 0.4171806 0.5217459

Random intercepts model

random_intercepts_model <- brms::brm(
  brms::bf(CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (1|Visit) + (1|Child.ID) + (1|Diagnosis)),
  data = data,
  file = 'data/w6/random_intercepts',
  chains = 2,
  cores = 2,
  seed = 112)

Random Intercepts Results

## Warning: There were 160 divergent transitions after warmup. Increasing
## adapt_delta above 0.8 may help. See
## http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (1 | Visit) + (1 | Child.ID) + (1 | Diagnosis) 
##    Data: data (Number of observations: 352) 
##   Draws: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 2000
## 
## Group-Level Effects: 
## ~Child.ID (Number of levels: 61) 
##               Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sd(Intercept)     0.56      0.06     0.46     0.68 1.01      329      632
## 
## ~Diagnosis (Number of levels: 2) 
##               Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sd(Intercept)     1.44      1.10     0.07     4.02 1.01      902      795
## 
## ~Visit (Number of levels: 6) 
##               Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sd(Intercept)     0.17      0.11     0.06     0.43 1.01      474      658
## 
## Population-Level Effects: 
##                    Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept              0.03      1.33    -2.90     2.86 1.00      777      799
## MOT_MLU                0.37      0.05     0.27     0.48 1.00      946     1313
## DiagnosisTD           -0.28      1.85    -4.18     3.63 1.01      819      842
## DiagnosisASD:Visit     0.06      0.05    -0.03     0.17 1.00      463      472
## DiagnosisTD:Visit      0.30      0.05     0.20     0.40 1.01      458      459
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.41      0.02     0.38     0.45 1.00     1146     1117
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate  Est.Error      Q2.5     Q97.5
## R2 0.8066044 0.01062774 0.7840735 0.8253711

Random Slopes Model

random_slopes_model <- brms::brm(
  brms::bf(CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (MOT_MLU|Diagnosis) + (MOT_MLU + Diagnosis|Visit) + (MOT_MLU + Diagnosis + Visit|Child.ID)),
  data = data,
  file = 'data/w6/random_slopes',
  chains = 2,
  cores = 2,
  seed = 112)

Random Slopes Results

## Warning: There were 327 divergent transitions after warmup. Increasing
## adapt_delta above 0.8 may help. See
## http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (MOT_MLU + Diagnosis | Visit) + (MOT_MLU | Diagnosis) + (MOT_MLU + Diagnosis + Visit | Child.ID) 
##    Data: data (Number of observations: 352) 
##   Draws: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 2000
## 
## Group-Level Effects: 
## ~Child.ID (Number of levels: 61) 
##                            Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)                  0.50      0.24     0.05     0.98 1.00      220
## sd(MOT_MLU)                    0.21      0.06     0.11     0.35 1.01      144
## sd(DiagnosisTD)                0.54      0.16     0.23     0.87 1.01      213
## sd(Visit)                      0.09      0.02     0.06     0.13 1.01      135
## cor(Intercept,MOT_MLU)        -0.62      0.33    -0.96     0.28 1.02      112
## cor(Intercept,DiagnosisTD)     0.26      0.37    -0.52     0.86 1.03      139
## cor(MOT_MLU,DiagnosisTD)      -0.68      0.18    -0.94    -0.25 1.01      452
## cor(Intercept,Visit)          -0.49      0.32    -0.93     0.26 1.03       88
## cor(MOT_MLU,Visit)             0.37      0.27    -0.16     0.83 1.02      307
## cor(DiagnosisTD,Visit)        -0.52      0.24    -0.92     0.00 1.01      371
##                            Tail_ESS
## sd(Intercept)                   388
## sd(MOT_MLU)                     442
## sd(DiagnosisTD)                 573
## sd(Visit)                       241
## cor(Intercept,MOT_MLU)          230
## cor(Intercept,DiagnosisTD)      393
## cor(MOT_MLU,DiagnosisTD)        839
## cor(Intercept,Visit)            270
## cor(MOT_MLU,Visit)              734
## cor(DiagnosisTD,Visit)          801
## 
## ~Diagnosis (Number of levels: 2) 
##                        Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)              1.88      1.65     0.08     6.16 1.01     1007
## sd(MOT_MLU)                0.56      0.49     0.03     1.80 1.01      333
## cor(Intercept,MOT_MLU)    -0.04      0.57    -0.96     0.96 1.01     1402
##                        Tail_ESS
## sd(Intercept)              1028
## sd(MOT_MLU)                 863
## cor(Intercept,MOT_MLU)     1020
## 
## ~Visit (Number of levels: 6) 
##                            Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)                  0.20      0.22     0.01     0.72 1.01      608
## sd(MOT_MLU)                    0.08      0.06     0.01     0.22 1.01      366
## sd(DiagnosisTD)                0.35      0.25     0.09     0.97 1.00      668
## cor(Intercept,MOT_MLU)        -0.34      0.49    -0.97     0.78 1.01      712
## cor(Intercept,DiagnosisTD)     0.02      0.49    -0.88     0.88 1.00      569
## cor(MOT_MLU,DiagnosisTD)      -0.17      0.42    -0.87     0.72 1.01     1199
##                            Tail_ESS
## sd(Intercept)                   770
## sd(MOT_MLU)                     549
## sd(DiagnosisTD)                 908
## cor(Intercept,MOT_MLU)         1082
## cor(Intercept,DiagnosisTD)      976
## cor(MOT_MLU,DiagnosisTD)        943
## 
## Population-Level Effects: 
##                    Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept              0.61      1.86    -3.67     4.39 1.01      423      883
## MOT_MLU                0.34      0.36    -0.45     1.01 1.01      162      123
## DiagnosisTD           -0.97      2.63    -6.67     5.12 1.01      465      866
## DiagnosisASD:Visit     0.06      0.07    -0.09     0.19 1.00      602      525
## DiagnosisTD:Visit      0.29      0.12     0.06     0.52 1.01      723      518
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.34      0.02     0.31     0.38 1.00      362     1096
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate  Est.Error      Q2.5     Q97.5
## R2 0.8683146 0.01016479 0.8467115 0.8852524

Nested Grouping Model

nested_grouping_model <- brms::brm(
  brms::bf(CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (MOT_MLU|Diagnosis) + (MOT_MLU + Diagnosis|Visit|Child.ID)),
  data = data,
  file = 'data/w6/nested_grouping',
  chains = 2,
  cores = 2,
  seed = 112)

Nested Grouping Results

## Warning: There were 254 divergent transitions after warmup. Increasing
## adapt_delta above 0.8 may help. See
## http://mc-stan.org/misc/warnings.html#divergent-transitions-after-warmup
##  Family: gaussian 
##   Links: mu = identity; sigma = identity 
## Formula: CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (MOT_MLU | Diagnosis) + (MOT_MLU + Diagnosis | Visit | Child.ID) 
##    Data: data (Number of observations: 352) 
##   Draws: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 2000
## 
## Group-Level Effects: 
## ~Child.ID (Number of levels: 61) 
##                            Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)                  0.65      0.26     0.15     1.14 1.00      379
## sd(MOT_MLU)                    0.30      0.07     0.18     0.44 1.00      289
## sd(DiagnosisTD)                0.54      0.15     0.26     0.86 1.00      443
## cor(Intercept,MOT_MLU)        -0.82      0.21    -0.99    -0.20 1.01      167
## cor(Intercept,DiagnosisTD)     0.41      0.36    -0.38     0.95 1.00      189
## cor(MOT_MLU,DiagnosisTD)      -0.77      0.17    -0.98    -0.35 1.01      311
##                            Tail_ESS
## sd(Intercept)                   326
## sd(MOT_MLU)                     347
## sd(DiagnosisTD)                 730
## cor(Intercept,MOT_MLU)          259
## cor(Intercept,DiagnosisTD)      357
## cor(MOT_MLU,DiagnosisTD)        734
## 
## ~Diagnosis (Number of levels: 2) 
##                        Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)              1.80      1.51     0.07     5.81 1.00      996
## sd(MOT_MLU)                0.50      0.45     0.03     1.65 1.01      714
## cor(Intercept,MOT_MLU)    -0.05      0.60    -0.96     0.97 1.00     1182
##                        Tail_ESS
## sd(Intercept)               868
## sd(MOT_MLU)                 967
## cor(Intercept,MOT_MLU)      584
## 
## Population-Level Effects: 
##                    Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept              0.36      1.66    -3.18     3.94 1.00     1086      849
## MOT_MLU                0.36      0.27    -0.31     0.93 1.01      738      846
## DiagnosisTD           -1.09      2.47    -7.04     3.93 1.00      931      618
## DiagnosisASD:Visit     0.07      0.02     0.03     0.11 1.00     1808     1252
## DiagnosisTD:Visit      0.29      0.02     0.25     0.33 1.00     2153     1335
## 
## Family Specific Parameters: 
##       Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## sigma     0.39      0.02     0.36     0.43 1.01     1238     1089
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate  Est.Error      Q2.5     Q97.5
## R2 0.8234624 0.01069002 0.7998395 0.8431738

Estimation of multiple parameters

  • The first formula implicitly corresponds to the \(\mu\) parameter
    • Required by all brms families
    • Must have the desired outcome variable as DV
  • Each subsequent formula can correspond to any of the other (non-\(\mu\)) distributional parameters in the family
    • Since our family is gaussian, the second formula will have sigma as DV
multiple_parameters_model <- brms::brm(
 brms::bf(
    CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (MOT_MLU|Diagnosis) + (MOT_MLU + Diagnosis|Visit) + (MOT_MLU + Diagnosis + Visit|Child.ID),
    sigma ~ MOT_MLU + Diagnosis + (Diagnosis|Visit)),
  data = data,
  file = 'data/w6/multiple_parameters',
  chains = 2,
  cores = 2,
  seed = 112)

Multiple Parameters Estimation Results

##  Family: gaussian 
##   Links: mu = identity; sigma = log 
## Formula: CHI_MLU ~ MOT_MLU + Diagnosis + Diagnosis:Visit + (MOT_MLU | Diagnosis) + (MOT_MLU + Diagnosis | Visit) + (MOT_MLU + Diagnosis + Visit | Child.ID) 
##          sigma ~ MOT_MLU + Diagnosis + (Diagnosis | Visit)
##    Data: data (Number of observations: 352) 
##   Draws: 2 chains, each with iter = 2000; warmup = 1000; thin = 1;
##          total post-warmup draws = 2000
## 
## Group-Level Effects: 
## ~Child.ID (Number of levels: 61) 
##                            Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)                  0.50      0.21     0.11     0.94 1.03       83
## sd(MOT_MLU)                    0.20      0.06     0.10     0.33 1.10       18
## sd(DiagnosisTD)                0.48      0.16     0.21     0.79 1.01       58
## sd(Visit)                      0.09      0.02     0.06     0.13 1.01      161
## cor(Intercept,MOT_MLU)        -0.57      0.32    -0.95     0.22 1.09       19
## cor(Intercept,DiagnosisTD)     0.24      0.38    -0.54     0.85 1.04       44
## cor(MOT_MLU,DiagnosisTD)      -0.72      0.20    -0.97    -0.24 1.03       67
## cor(Intercept,Visit)          -0.50      0.29    -0.91     0.13 1.02       59
## cor(MOT_MLU,Visit)             0.41      0.26    -0.14     0.85 1.01       94
## cor(DiagnosisTD,Visit)        -0.51      0.25    -0.90     0.04 1.01      128
##                            Tail_ESS
## sd(Intercept)                   203
## sd(MOT_MLU)                     106
## sd(DiagnosisTD)                 195
## sd(Visit)                       339
## cor(Intercept,MOT_MLU)          135
## cor(Intercept,DiagnosisTD)      186
## cor(MOT_MLU,DiagnosisTD)        118
## cor(Intercept,Visit)            221
## cor(MOT_MLU,Visit)              220
## cor(DiagnosisTD,Visit)          336
## 
## ~Diagnosis (Number of levels: 2) 
##                        Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS
## sd(Intercept)              2.62      2.73     0.05     9.32 1.03       61
## sd(MOT_MLU)                0.55      0.59     0.03     2.06 1.01      208
## cor(Intercept,MOT_MLU)    -0.05      0.56    -0.95     0.93 1.02      329
##                        Tail_ESS
## sd(Intercept)                76
## sd(MOT_MLU)                 370
## cor(Intercept,MOT_MLU)      686
## 
## ~Visit (Number of levels: 6) 
##                                        Estimate Est.Error l-95% CI u-95% CI
## sd(Intercept)                              0.24      0.22     0.01     0.75
## sd(MOT_MLU)                                0.08      0.06     0.01     0.23
## sd(DiagnosisTD)                            0.30      0.21     0.07     0.87
## sd(sigma_Intercept)                        0.14      0.13     0.01     0.48
## sd(sigma_DiagnosisTD)                      0.28      0.23     0.01     0.86
## cor(Intercept,MOT_MLU)                    -0.41      0.49    -0.99     0.69
## cor(Intercept,DiagnosisTD)                 0.11      0.49    -0.88     0.90
## cor(MOT_MLU,DiagnosisTD)                  -0.19      0.43    -0.89     0.74
## cor(sigma_Intercept,sigma_DiagnosisTD)    -0.01      0.58    -0.95     0.94
##                                        Rhat Bulk_ESS Tail_ESS
## sd(Intercept)                          1.02       89      355
## sd(MOT_MLU)                            1.02       96      404
## sd(DiagnosisTD)                        1.02      175      362
## sd(sigma_Intercept)                    1.01      145      468
## sd(sigma_DiagnosisTD)                  1.02      140       84
## cor(Intercept,MOT_MLU)                 1.04       58       28
## cor(Intercept,DiagnosisTD)             1.02      152      196
## cor(MOT_MLU,DiagnosisTD)               1.02      397      622
## cor(sigma_Intercept,sigma_DiagnosisTD) 1.01      119      264
## 
## Population-Level Effects: 
##                    Estimate Est.Error l-95% CI u-95% CI Rhat Bulk_ESS Tail_ESS
## Intercept             -0.34      3.05    -9.90     3.90 1.05       61       17
## sigma_Intercept       -1.87      0.35    -2.56    -1.17 1.01      411      746
## MOT_MLU                0.38      0.37    -0.25     1.40 1.01      126      148
## DiagnosisTD            0.53      5.47    -6.44    18.70 1.03       56       16
## DiagnosisASD:Visit     0.08      0.07    -0.05     0.21 1.01      174      250
## DiagnosisTD:Visit      0.29      0.10     0.09     0.49 1.01      273      323
## sigma_MOT_MLU          0.21      0.09     0.03     0.39 1.01      365      583
## sigma_DiagnosisTD     -0.16      0.18    -0.63     0.14 1.02      269      219
## 
## Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
## and Tail_ESS are effective sample size measures, and Rhat is the potential
## scale reduction factor on split chains (at convergence, Rhat = 1).
##     Estimate  Est.Error      Q2.5     Q97.5
## R2 0.8621564 0.01224518 0.8348675 0.8826969

Prior Elicitation

Why Gaussian distributions?

  • Most natural expression of our state of ignorance
    • If we only assume that a variable has finite variance, the Gaussian distribution is the shape that can be realized in the largest number of ways
    • Least surprising and least informative assumption to make.
    • “If you don’t think the distribution should be Gaussian, then that implies that you know something else (…), something that would improve inference”. (McIlreath, pp. 75–76)

Simple bayesian regression

## CHI_MLU ~ 1 + MOT_MLU
brms::prior_summary(simple_model)
##                   prior     class    coef group resp dpar nlpar lb ub
##                  (flat)         b                                    
##                  (flat)         b MOT_MLU                            
##  student_t(3, 1.9, 2.5) Intercept                                    
##    student_t(3, 0, 2.5)     sigma                                0   
##        source
##       default
##  (vectorized)
##       default
##       default

Setting Simple Model Priors

simple_model_priors <- c(prior(normal(0, .5), class = b),
                         prior(normal(0, 1), class = sd))
simple_model_with_priors <-
  update(simple_model,
         prior = simple_model_priors,
         file = 'data/w6/simple_model_with_priors',
         sample_prior = 'only',
         chains = 2)

Simple Model Prior Effect

Simple Model with Lognormal Priors

simple_model_lognormal_prior <-
  c(prior(lognormal(0, 1), class = b))
simple_model_with_lognormal_prior <-
  update(simple_model,
         prior = simple_model_lognormal_prior,
         file = 'data/w6/simple_model_with_lognormal_prior',
         sample_prior = 'only',
         chains = 2)

Simple Model Lognormal Prior Effect